Okay, so welcome back everybody to interventional medical image processing.

We started talking a bit about different pre-processing algorithms and what we wanted to aim at are algorithms that are fast in particular.

And so far we have seen that we can cut down on the complexity of algorithms and then end up with several algorithms that have the ability to be executed very fast.

Later we also want to interpret images and in order to interpret images we will make use of low level descriptors that we then can compose to different structures and extract information from the image.

So later we may want to detect certain features of the image and often what we do is we do some kind of processing to simplify the information extraction.

One thing that will be very important are edges. So edges and gradients. And edges and gradients are very useful in order to detect important points in an image and we will talk today about something which is called the structure tensor and why the structure tensor is interesting to extract features from an image.

So why are edges and gradients interesting? Well, if you think about an image you are most of the time the points that will pop up in your eye immediately are changes in intensity.

So if you for example think of a slice image what you will note is that if you have some kind of let's say you have a slice of a patient.

I have an example here. So let's say you have a slice of a patient image so this is a CT slice and this one is contrast enhanced. You will see that you have homogeneous areas in the image so there's homogeneous areas like here and then there are certain changes in intensity.

And these changes in intensity are in particular interesting. And on the right hand side you see the gradient image. So the gradient image and here the gradient norm is denoted in color shows you the important features of the image.

So you can see that for example the table here appears very bright and furthermore the outline of the patient so at the boundary of the patient you have a strong change in gradient so there is a change in the intensity and this pops up in the gradient image.

So the gradient image is related to edges and edges are interesting because let's say you want for example to detect the outline of an organ so you want to determine the surface of the liver or something like this.

And then you can see that the organ has an edge, an intensity edge surrounding and you can use this edge information to determine the outline. Also here you can see that the different organs they are outlined by edges.

Then what can we further see? There's a very strong edge from air to the patient because the patient is composed mostly of water so soft tissues and you have a strong edge from air to soft tissue and there's also a very strong edge again here.

Why do we have the strong edge here? Can you imagine? Yes?

Well this appears dark in this image so what could be here? There's another strong edge here. Can you see the strong edge here down here? What's this?

Yeah so this is part of the spine bone and the bone is much denser than the soft tissue so you have a very strong edge here. But here you also have a very strong edge.

Yeah it's air in the colon. Exactly. So we have strong edges in this image between air and soft tissue and again we have an edge between air and soft tissue here so this is not a bone.

But the height of this edge here is approximately the same as in the area down here. You can see that? So in the gradient image we see the edge but here this is the norm of the gradient so it doesn't tell us the direction of the gradient because we're only looking at the length of the gradient vector.

And therefore we cannot differentiate anymore between the interface between air and soft tissue or between soft tissue and bone. They appear very similar in this image.

So you also need to think about what is actually shown in this image that you can understand the kind of information that is displayed.

So this is not a patient that has two spines but he just has some air in the colon. Okay good. So what do we have to understand about the gradient?

Well in images we have two or three dimensions and in general the gradient is the derivative with respect to the different orientations. So here in the example in equation one we have a 2D gradient because we also have 2D images in the following.

And here is the derivative of the image with respect to x and the derivative of the image with respect to y. And in the following we will call this fx and fy so component wise.

And now the other image that I showed on the actually on the next slide was showing the gradient norm. So the gradient is generally a vector quantity because it has an orientation and a strength.

And if we want to display a vector quantity in a single image we have different ways of reducing this in order to show it. So one way would be to show the orientation.

So we show the color coded angle of the gradient or we can look at the norm then we have the strength of the gradient or we can also do a component wise visualization.

So we could only look at fx and fy here. So this would also be because these are the two components of the vector they would also be an entire image those two components.

Generally we have certain trade off that we have to deal with generally in CT images or also in other imaging modalities.

You can have for example in CT you will end up with very noisy images if you try to achieve a low patient dose.

So if you go down with the dose the number of x-rays that you radiate the patient with you will end up with a higher noise level.

And the other thing that you have to keep in mind is the diagnostic task. So if you are interested in a very low contrast so in a very little jump in terms of intensity

then you need to achieve a very good low contrast visibility and there it will be hard to see the low contrast if you have a lot of noise.

So if you are looking for an intensity difference and let's say the kind of difference that you are looking at is 10 hz units.

So the kind of function the difference that you are looking at is this.

But then the signal is overlaid with a noise of let's say 30 hz units then you end up with something like this and then you have maybe a function like this.

So in this kind of function it will be very difficult to see your 10 hz units difference.

And generally your noise is of very high frequency so this is the other point if you now run edge detection on this one here then you will detect a lot of edges because the noise will cause a lot of edges.

So now the thing is can we do some processing techniques in order to help us with the image interpretation.

And one thing that you may see here already is typically those low contrasts that you are looking for they don't change from pixel to pixel.

Typically they have low spatial frequencies so they have some certain extent and you can use that in order to suppress the noise and you can always suppress image noise by smoothing, by blurring.

But this also means that the image resolution goes down but sometimes you can sacrifice resolution in order to improve the noise contrast.

The contrast to noise level.

Okay good.

So how do we do now this how do we calculate those gradients.

So if our image would be a continuous function then we could compute the derivative with respect to the two orientations and we had our gradient.

But typically our images are discrete.

So we need some kind of discrete approximation of the gradient.

And so what we want to do is we want to detect those high frequency those high changes in intensities and we want to compute this discrete gradient somehow from this image.

A general couple of general remarks is the edge that we have when we talk about gradients and edges so the edge is typically a set of is a structure in the image that has a set of oriented gradients and those gradients point into an orthogonal direction with respect to the edge.